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ABSTRACT 

The final evaluation report for the Colorado City 
Bilingual Education Program is presented in this paper. The project 
was directed to kindergarten children from low-income disadvantaged 
homes; of 63 participants, 53 were Spanish-speaking children and 10 
were monolingual English-speaking children. Project staff consisted 
of a bilingual teacher, a native English-speaking teacher, 3 
bilingual teacher aides, and a bilingual director. The project, 
located at Kelley Elementary School, was evaluated in terms of its 
administrative component, instructional program, and community 
support. Pre- and post-test data were obtained using the following 
instruments: (1) Short Test of Educational Abilities, (2) Peabody 

Picture Vocabulary Test, (3) Tests of Easic Experiences, and (4) 
Check-List of English Words. The greatest weakness of the project was 
the evaluation design used, which was not an integral part of the 
instructional program. The greatest strength was the project* s 
uniqueness of size, which permitted individual contact with all 
personnel. Included in the report are tables of test results. (NQ) 
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he final evaluation report contained Herein for the Color: 

_ity Bilingual Education Program is intended to dembnstrati 
degree of accomplishment of stated objectives as described in 
the grant application for Title VII funds- Evaluation was per- 
formed as described in the evaluation design. It should be 
noted, however, that this evaluator does not consider the stan- 
dardized instruments implemented as true measures cf educational 
accomplishment nor do they reflect the outstanding duties per- 
formed by the instructional and administrative staff. As an 
outside, independent agent, it was my unbiased observation that 
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written or formal evaluation instruments, 
ever, has been in close contact with project personnel through- 
out the fiscal year and is awe 
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Four meetings were held during 
director, evaluation consultant 
the program and its effectivene 
of the various components were 
steps taken to implement them. 
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The proposal stated that the project would be located in 
Kelley Elementary School, LH35 Elm Street, Colorado City, 
would be selected on the basis of bilingual abilities and 
certification. At least one teacher was to be bilingual, 
aides employed for the program were to be bilingual. 
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One teacher, Miss Rita. Naredo , is of Cuban descent, and is 
totally bilingual. She learned English as a second language 
and has mastered both fluency and pronunciation to an impeccable 

degree . 
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The other teacher, Mrs. Carol Fish, is a. native English speaker 
who learned Spanish as a second language. Her fluency and pro- 
nunciation have been an asset to her teaching. 

The three teacher’s aides were bilingual and performed their 
duties in an extremely successful manner. Teamwork and dedica- 
tion were observed to be significant contributors to the instruc 
tional program. 

The director of the program, Mrs. Nell Price, is bilingual and 
fully certified. Her educational experience in Spanish totals 
nine years in South America teaching Spanish and one year teach- 
ing reading to Mexican-American children in Colorado City. Her 
involvement through leadership and demonstration contributed 
significantly to the success of the rrogram. 

Office files contain the following information which indicate 
degrees of accomplishment of stated objectives: 

An estimated sixty (60) children were to be enrolled 
' in a ratio of one English -speaking child per each 
five Spanish speaking children. The actual enroll- 
ment was sixty-three (63) children of which fifty- 
three (53) were Spanish speaking and ten (10) were 
monolingual English speaking. 

Monthly reports to the assistant superintendent of schools were 
submitted and periodic reports to the school board were made. 

The following miscellaneous information relevant to managerial 
duties are on file: 



Average Daily Attendance (ADA) 88% 

Home visits by Community Liaison Aide 367 

Conferences Attended by Staff 

a. Director 2 

b. Teachers . 9 

Visits by Staff to Other Districts (Projects) 2 

Community Relationships Presentations by 

Director to Service Clubs, etc. 15 

Job Descriptions All staff 

Units of Study 10 Spanish 

9 English 

Advisory Committee Meetings 2 



INSTRUCTIONAL 

The instructional program was directed to the kindergarten-aged 
child. The children were primarily from low-income disadvantaged 
homes in which only one. language was usually spoken. 
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Evaluation of this component presented several difficulties 
which most likely affected the test results. Outstanding among 
these is the fact that these children had never been exposed to 
testing of any kind and the mere mechanics of the instruments 
led to much confusion. This was most significant during the 
pre-test period when children were just beginning to adjust to 
a school environment. Other problems were locating a suitable 
room for testing small groups of children selected for the 
sample. Because of the selection of a random sample comprised 
of children from both classes, some discomfort and apprehension 
were observed in the children. Illnesses among the children for 
the post-test sessions prevented a pre- post-test administration 
yielding a smaller number of testees. The vocabulary of the 
test was often too difficult for the children and required some 
translation into the regional dialect in order to assure con- 
cept comprehension. This was especially true of the PPVT and 
TOBE. Although proctors were used during the TOBE pre-test, it 
was difficult to keep children from "copying" from each other 
due to the complexity of the test construction. It is therefore 
concluded by this evaluator that pre-test data is unreliable and 
does not yield accurate comparison information. 

In the Short Test of Educational Abil it ies (STEA), the Spatial 
Relations section proved to be too difficult for the children 
to perform. It became necessary to divide the group into three 
snail clusters where the. evaluator and two proctors could pro- 
vide more individualized instructions on how to do it. This 
was a laborious time consuming effort and the results probably 
inaccurate as far as a true measure of the children's learning 
abilities . 

The P eabo dy Pic ture Voc abulary Test (PPVT) was translated into 
a regional ri.1au.ect. This" was obtained through group effort .in- 
volving the director, evaluator, evaluation consultant, auditor, 
and aides. The latter are products of the children's enyironment. 
In spite of the ranslation, the results are probably still not 
an accurate measure of each child's potential. Many variables 
can influence responses such as administration of tne test by 
the director and the community liaison aide, removing a child 
from the classroom to an alien environment (office), and the 
discomfort observed as a result of being away from the peer group 
for testing. 

The Tests of Basic Experiences (TOBE) are not correlated to the 
curriculum. Recognition is granted that this is not a criterion 
reference, test; but, the results do not really coincide with the 
instructional, activities in so far as yielding effective measures 
of educational accomplishment . A translation into Spanish was 
obtained from the publisher for . the various tests; but, the 
vocabulary and concepts to elicit responses are considered to be 
inappropriate for the age level and background of the children. 

The audio tapes of each student were extremely difficult to 
decipher. Identifying proivmciation errors, vocabulary, etc., 
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were difficult to understand because of the children’s low voice 
level. It was possible, however, to isolate pronunciation errors 
such as sh, ch, d, th, and some vowels not common to the Spanish 
language . 

The picture to which the children responded was a farm scene taken 
from the Peabody Language Development Kit. It contained many pic- 
tures of familiar animals but it was the concurrence of all in- 
volved that it was too "busy", or distracting. It caused children 
to often respond more than once to the same animal or activity. 
There was evidence of language growth, however, on the post- 
recording session where notable progress was observed in sentence 
patterns, and an increase in fluency. This may have been due to 
social maturity and a more at ease countenance on the part of the 
children. Fewer errors in pronunciation were evidenced also. No 
effort was made to measure individual, vocabulary growth because of 
the insurmountable task of isolating the various components which 
could be obtained. 

Plans for the next project year, 1971-72, are to select a picture 
of fewer activities and objects and ask the child to respond to 
a specified number. The picture will also contain some of the 
basic speech sounds which are not common to their native language 
thus permitting a more critical analysis of the data to be deter- 
mined . 

The Check-List of E nglish Words administered by the teachers has 
proven to be of great assistance to teachers in directing their 
instructional activities to remedy vocabulary problems. They, 
again, are subject to teacher judgment and may require more in- 
service training or the employment of a linguist to make more 
professional evaluation of language performance. 

# 

The lists of minimal pair-words with pictures developed locally 
were also beneficial to teachers in assessing language skills. 
Although not standardized, the information gleaned from such an 
instrument is a tangible which teachers are able to implement in 
their instructional activities. 

Physical maturity was measured through the use of locally devel- 
oped tasks. These involved stacking of six blocks into a pyra- 
mid, stringing fifteen plastic beads onto a shoe lace, and com- 
pletion of an eight piece f or mb oar d puzzle. Results are described 
in a chart which is enclosed in the test results section. This 
activity was of questionable value to the teachers since no 
vocabulary measurement is involved. 

It is hoped by this evaluator that the preceding comments will 
enlighten the reader to better understand the disparity among 
the test scores derived. The concern concluded from this exper- 
ience is that standardized instruments for a project of this size 
arc inappropriate and do not recognize the true instructional 
values which are observed. 
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The irue value of this endeavor is that modifications in the 
evaluation design are necessary and a search for more suitable 
instruments should be undertaken. Certainly, in-service acti- 
vities for the instructional staff are in order. The develop- 
ment of their skills in stating their activities in performance 
objective terms with evaluation as an integral part of their 
plans would strengthen the instructional process. 

RECOGNITION OF STRENGTHS AND WEAKNESSES 

The following strengths were observed by this evaluator as 
having significant impact on the success of the program: 

Administration The total administrative staff inclu- 
ding the school board, superintendent, assistant 
superintendent, principal and director are mutually 
supporting the program in word and deed. That the 
program will be continued beyond the five-year grant 
period is evidenced in long-range objectives which 
are presently being designed. Among these is a 
building program which gives consideration to the 
bilingual component. The school faculty strongly 
supports the project through visitation of the 
classrooms and comments. 

Instruction The two teachers, aides, and secretary 
are truly dedicated to their work. Evaluation 
instruments could never measure this dedication as 
accurately as it is observed. They are willing to 
try new methods and vary their’ instruction to main- 
tain high interest level. To this evaluator, the 
best way to measure this is to observe children as 
they arrive at school during the day and at the end 
of the day and observe their composure. Children 
are always happy and smiling and enjoy the activi- 
ties, Even the testing was fun to them and elicited 
comments such as "When do I do that?" or "Let me be 
next . " 

Communi ty The community supports the program as 
evidenced through monetary contributions for snacks, 
etc. Contributions from service clubs for snacks 
totalJed $285. No estimate can be made of che mone- 
tary amount for refreshments, clothing, shoes, and 
play clothes donated to the program 

Comments in the community are favorable in all 
respects. A recognition is granted that bilingual 
instruction is good and necessary. 

The greatest weakness in the opinion of this evaluator is the 
evaluation design. Evaluation should be an integral part of 
instruction and not necessarily from an extraneous source. A 
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checklist form for teachers to use on a weekly or monthly basis 
describing each student’s performance would be more beneficial 
and would yield data to compare with selected standardized in- 
struments . 

SUMMARY 



PPVT (Spanish ) 

The pre-test — post-test scores derived from the Spanish admini- 
stration of the test reveal that on the average five months of 
mental age were gained by those students tested. 

Pre-test Average M. A. 3-3 

Post-test Average M. A. 3-8 

Average Gain in M. A. 0-5 

The scores derived from the English administration indicate an 
average growth in mental age of one year and seven months. 

Pre-test Average M. A. 2.8 

Post-test Average M. A. V. 4 . 3 
Average Gain in M. A. 1.7 

This would seem to indicate that more time was devoted to English 
instruction than to Spanish. It was this evaluator’s observation 
however, and also reflected in lesson plans that the ratio of 
English-Spanish instruction was approximately 60-40 respectively. 

It is interesting to note, however, that the five mono-lingual 
English speaking students made little or no gains in Spanish M. A 
but gained significantly in English M. A. (Refer to students 6, 
7, 13, 30, 31 on test data page) 

At the same time, the Spanish-speaking students performed better 
in English than in Spanish. It is this evaluator’s opinion that 
this is related to the possibility that parents perhaps encourage 
their children to learn more English than Spanish plus the fact 
that the children most likely wish to earn the praise of the 
teacher and peers by learning the second language better. 

STEA 



The results derived from the administration of the STEA are 
significantly higher than those of the PPVT . This is probably 
due to ’’copying” which was alluded to earlier in this report . 
Again reference is made to the monolingual children who scored 
higher on this test than on the PPVT. The results of this test 
are considered to be invalid by this evaluator for reasons dis- 
cussed earlier. 

TOBE (Language, Math, Social Studies, Science) 



Although the complete battery of tests does not correlate with 
the instructional program to any great degree, it is interesting 
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to note the similarity cf pre- and post-test scores on all sub 
tests . 



Spanish Percentile Pre-test Average 

Post-test Average 

English Percentile Pre-test Average 

Post-test Average 





Social 


English 


Studies 


9.4 


8.5 


36.1 


34.0 


22. C 


16.8 


31.5 


38.2 



Math 


Science 


13.5 


15.5 


36.0 


29.5 


22.7 


11.6 


31.8 


30.9 



It would appear from a cursory examination of the data that the 
greatest gains were made in Spanish although pre-test scores were 
higher in English. This seems to be more consistent with the 
60-40 ratio mentioned earlier if indeed the tests measure basic 
experiences and concept development. 



CONCLUSIONS 

The evaluation of the various components is a very difficult task 
in that suitable standardized instruments are not available. 

Although the data obtained from this effort is conflicting in 
some areas, sufficient evidence is demonstrated to reveal strengths 
in the instructional and managerial components. It is this eval- 
uator’s opinion that the strongest part of this program is the 
uniqueness of size which perm.its individual contact with all per- 
sonnel and the spirit of cooperation and dedication which prevails. 

A great deal of verbal evaluation is carried on between the 
teachers and the director and it has been this evaluator’s good 
fortune to have been an integral part of these activities. The 
recording of these small sessions would be ideal but would likely 
hinder the openness and mutual sharing which exists. These off- 
the-record conferences are much more beneficial than data collection 
which often goes unnoticed. 

RECOMMENDATIONS 

The following recommendations are made regarding the 1971-72 
project year. 

1. That a more refined evaluation design be developed 
which will more closely involve instructional per- 
sonnel. 

2. That intensive in-service training be undertaken 
regarding developing instructional or performance 
objectives with strong emphasis on continuous 
evaluation by the teachers. 

3. That individual checklists of student performance 
be developed on a weekly, biweekly or monthly 
basis depending upon the number of items and 
completion time required. 

4. That the evaluator submit a written report at the 
conclusion of each visitation to the director 
describing observations and duties performed and 
that this information be discussed with the staff. 
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